Skip to content

Read a media value's content with ['base64'] - #5608

Open
aaronvg wants to merge 1 commit into
aaron/thread-error-log-linefrom
aaron/media-content-path
Open

aaronvg wants to merge 1 commit into
aaron/thread-error-log-linefrom
aaron/media-content-path

Conversation

@aaronvg

@aaronvg aaronvg commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

A query describes an image, audio, PDF or video value without its content, as {"$media", "mime", "base64_len"}, so a result of a thousand runs is not a thousand pictures. A path that ends in ['base64'] on a media value now reads that one value's content as base64 text, from the value's own blob or from the child blob a large content lives in. Media that names a URL or a file holds no content and reads as missing.

Studio uses this path to show images and PDFs in span details (BoundaryML/bcs#263).

Part of a stack; review the PRs in order.

🤖 Generated with Claude Code


Note

Medium Risk
Changes captured-value navigation and SQL query semantics for media; behavior is well-tested but touches telemetry/query paths that Studio depends on.

Overview
baml query can now load a single media value’s payload while default rendering still shows only lightweight descriptors ($media, mime, base64_len or url).

Value navigation treats base64 as a special key on media objects: paths like input_args['small']['_data']['base64'] return the stored base64 text (inline or from an external CAS child blob), respect blob budgets, and yield missing for URL/file-backed media or other keys such as mime. Documentation for baml query describes this behavior.

Coverage adds unit tests in btel_reader and an end-to-end query test that asserts small/large images, URL-only media, descriptors, and SQL equality on the loaded content.

Reviewed by Cursor Bugbot for commit e64fa2e. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features
    • Querying a media value’s base64 content now returns its text for inline and separately stored image, audio, PDF, and video data. URL-backed media and unsupported media keys continue to return no value.
    • Media descriptors remain content-free; paths that end at base64 can read the content, while paths beneath it do not resolve.
  • Documentation
    • Clarified how media content is represented and accessed in queries.

@vercel

vercel Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
developer-docs Ready Ready Preview Nov 13, 2026 1:01am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Media values can now expose their content through the MEDIA_CONTENT navigation key. Inline payloads are returned directly; referenced payloads are loaded and validated. Tests cover missing paths, unavailable content, and query behavior.

Changes: Media Content Navigation

Layer / File(s) Summary
Resolve media content
baml_language/crates/btel_reader/src/value.rs, baml_language/crates/btel_reader/src/value/span.rs, baml_language/crates/btel_reader/src/value/span/tests.rs, baml_language/crates/baml_query_btel/tests/values.rs, baml_language/crates/baml_builtins2/keyword_docs/baml_keywords.yaml
The MEDIA_CONTENT key resolves inline content or loads and validates content from a referenced child blob. Tests cover supported and missing paths, unavailable content, descriptor rendering, and query filtering. Query documentation describes media content access.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant BtelQuery
  participant ValueStep as value::step
  participant MediaText as span::media_text
  participant ChildBlob
  BtelQuery->>ValueStep: Navigate to MEDIA_CONTENT
  ValueStep->>MediaText: Resolve media payload
  opt External payload
    MediaText->>ChildBlob: Load referenced content
    ChildBlob-->>MediaText: Return child content
  end
  MediaText-->>ValueStep: Return content or navigation result
  ValueStep-->>BtelQuery: Return navigation result
Loading

Suggested reviewers: 2kai2kai2, hellovai

Merge Risk: Minimal · up to 65875

This change lets queries read a media value's content as base64 text. Its behavior is covered by unit and query tests, and no merge-blocking risk was identified.

Security architecture risk: Low · up to 65875

Explicit queries can now return captured media content. Existing redaction, integrity checks, and read budgets remain in place, and no boundary bypass was identified. Access controls in the consuming application were not available for verification.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — At the inspected query layer, the exposure expands to captured media reachable through value navigation in the configured project's CAS. The change does not itself introduce a new network destination, arbitrary media-file read, or additional service credential. Remote consumer authorization and tenant isolation remain outside the inspected scope.

Trust Boundaries and Controls

  • observed — User-selected query paths reach the new payload read through the existing read-only query machinery. The SQL translator accepts only one query statement, and SQLite authorization denies non-read actions and privileged file or extension functions. These controls constrain execution authority but do not establish who may access a project's captured payloads.

Resilience and Maintainability Implications

  • observed — The existing string-scalar path returns complete text rather than applying the structured-render text cap. The new media path inherits this behavior. Default CAS reads are capped at 64 MiB, and default query output and duration budgets remain enabled; however, output size is checked after collecting a row, so that budget is not a strict peak-memory ceiling.
Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "Read a media value's content with ['base64']" matches the main change. The PR adds a query path ending in ['base64'] that reads media payloads as base64 text, including content in child CAS…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title "Read a media value's content with ['base64']" matches the main change. The PR adds a query path ending in ['base64'] that reads media payloads as base64 text, including content in child CAS blobs. The raw summary confirms the same behavior in the query docs, the btel_reader span navigation, and the new tests. The title is short, specific, and does not list files.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit finds a path to text,
Through blobs it hops, then checks what’s next.
Inline strings come into view,
Missing paths stay missing too.
The query nibbles base64 greens.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 688851ce-c73d-4928-951b-77eed2b3282e
Commits

Reviewing files that changed from the base of the PR and between 43e1232 and 65875ee.

Files selected for processing: 5

  • baml_language/crates/baml_builtins2/keyword_docs/baml_keywords.yaml
  • baml_language/crates/baml_query_btel/tests/values.rs
  • baml_language/crates/btel_reader/src/value.rs
  • baml_language/crates/btel_reader/src/value/span.rs
  • baml_language/crates/btel_reader/src/value/span/tests.rs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 2 remain after this review.

@aaronvg
aaronvg added this pull request to stack #5611 November 12, 2026 03:28
@cursor

cursor Bot commented Nov 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 384aef82-bf7d-474d-900f-8cdb11f862ba)

@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@github-actions

Copy link
Copy Markdown

Binary size checks failed

❌ 2 violations · ✅ 1 passed

⚠️ Please fix the size gate issues or acknowledge them by updating baselines.

Artifact Platform File Gzip Gated on Baseline Delta Status
✅ baml-cli Linux 🔒 65.0 MB 27.6 MB file 73.1 MB -8.1 MB (-11.0%) OK
❌ packed-program Linux 🔒 39.3 MB 16.2 MB file 29.2 MB +10.2 MB (+34.9%) FAIL
❌ bridge_wasm WASM 26.1 MB 🔒 7.7 MB gzip 5.7 MB +2.1 MB (+36.5%) FAIL

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.

Details & how to fix

Violations:

  • packed-program (Linux) file_bytes: 39.3 MB exceeds limit of 30.1 MB (exceeded by +9.2 MB, policy: max_file_bytes)
  • packed-program (Linux) file_delta_pct: +34.9% exceeds limit of 3.0% (exceeded by +31.9pp, policy: max_delta_pct)
  • bridge_wasm (WASM) gzip_bytes: 7.7 MB exceeds limit of 5.9 MB (exceeded by +1.9 MB, policy: max_gzip_bytes)
  • bridge_wasm (WASM) gzip_delta_pct: +36.5% exceeds limit of 3.0% (exceeded by +33.5pp, policy: max_delta_pct)

Add/update baselines:

.ci/size-gate/wasm32-unknown-unknown.toml:

[artifacts.bridge_wasm]
file_bytes = 26145178
gzip_bytes = 7727648

.ci/size-gate/x86_64-unknown-linux-gnu.toml:

[artifacts.packed-program]
file_bytes = 39318808
gzip_bytes = 16161060

Generated by cargo size-gate · workflow run

@aaronvg
aaronvg force-pushed the aaron/media-content-path branch from 02716df to 65875ee Compare November 13, 2026 00:11
@cursor

cursor Bot commented Nov 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 00e7ec0f-5b39-42db-b396-dad1eb028a0b)

@aaronvg
aaronvg deployed to boundary-tools-dev November 13, 2026 00:11 — with GitHub Actions Active
@aaronvg
aaronvg marked this pull request as ready for review November 13, 2026 00:11
@cursor

cursor Bot commented Nov 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 48578c75-8c98-48c7-a8fb-3d2b0831c70e)

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

A query describes an image, audio, PDF or video value without its content,
as `{"$media", "mime", "base64_len"}`, so a row of a thousand runs is not
a thousand pictures. A path that ends in `['base64']` on a media value now
reads that one value's content as base64 text, from the value's own blob
or from the child blob a large content lives in. Media that names a URL
or a file holds no content and reads as missing.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@aaronvg
aaronvg force-pushed the aaron/media-content-path branch from 65875ee to e64fa2e Compare November 13, 2026 00:59
@cursor

cursor Bot commented Nov 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: e4c48026-9b9c-4319-b471-589418dfa51b)

This branch was successfully deployed

2 active deployments
Preview – developer-docs — e64fa2e3 Deployed Nov 13, 2026 by vercel[bot]
boundary-tools-dev — e64fa2e3 Deployed Nov 13, 2026 by aaronvg via Docs / publish docs #14619
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant